How would you design a high-availability, horizontally scalable Go backend service?

Build a stateless service with all state in external stores, deploy N replicas behind a load balancer, use circuit breakers for downstream dependencies, and design for graceful degradation.

Stateless design principles

1
Stateless: all session state, cache, and persistent data lives in Postgres, Redis, or object storage
2
Connection pooling: pgxpool with MaxConns = (postgres_max_connections / replicas) - buffer
3
Health checks: /readiness (DB connected, dependencies available) and /liveness (process alive) for Kubernetes
4
Graceful shutdown: drain in-flight requests before exiting on SIGTERM
5
Zero-downtime deploy: rolling updates work because the service is stateless and backward-compatible

Circuit breaker and resilience

Scalability patterns

1
Horizontal scaling: deploy more replicas — works because service is stateless
2
Distributed locking: Postgres advisory locks or Redlock for critical sections across replicas
3
Caching: Redis for session data, hot DB queries, and rate limit counters
4
Bulkhead: separate goroutine pools for different request types to prevent cascade failures
5
Retry with exponential backoff and jitter for transient failures in downstream calls